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EXECUTIVE SUMMARY 


In this report, we describe disadvantaged students’ access to effective teaching in grades 4 
through 8 in 29 diverse school districts, using value-added analysis to measure effective 
teaching. Recent federal initiatives emphasize measuring teacher effectiveness and ensuring that 
disadvantaged students have equal access to effective teachers. These include Race to the Top, 
the Teacher Incentive Fund, and the flexibility policy for the Elementary and Secondary 
Education Act, which allows states to waive a number of provisions in exchange for a 
commitment to key reform principles (U.S. Department of Education 2009, 2012a). 

Federal efforts to promote the equitable distribution of effective teachers arise from 
concerns that disadvantaged students may have less access to effective teachers, thereby 
contributing to sizable achievement gaps for disadvantaged students (Reardon 2011; U.S. 
Department of Education 2012b). A growing body of research uses value-added analysis to 
measure teacher effectiveness and examine the extent to which disadvantaged students have 
access to effective teachers. Value added measures a teacher’s contribution to student learning, 
accounting for a student’s previous achievement level and background characteristics. Studies 
consistently find considerable variation in teacher effectiveness based on value-added measures 
(Nye et al. 2004; Rockoff 2004; Rivkin et al. 2005; Kane et al. 2006; Aaronson et al. 2007; 
Koedel and Betts 2009). In addition, there is evidence of better long-run outcomes for students 
taught by more effective teachers as measured by value added, including lower rates of teen 
pregnancy, increased likelihood of college attendance, and higher wages (Chetty et al. 2011). 

Given the importance of teachers in improving student achievement and concerns about 
unequal access to effective teachers (Jerald et al. 2009; Brown and Haycock 2011), more 
evidence on access to effective teaching is needed. This report focuses on access to effective 
teaching in 29 school districts over the 2008-2009 to 2010-2011 school years. 

The main findings are: 

• On average, disadvantaged students had less access to effective teaching in the 
29 study districts in grades 4 through 8. The magnitude of differences in effective 
teaching for disadvantaged and nondisadvantaged students in a given year was 
equivalent to a shift of two percentile points in the student achievement gap. 

Students eligible for a free or reduced-price lunch (FRL) experienced less effective 
teaching than non-FRL students on average within districts, with statistically 
significant differences of 0.034 standard deviations of student test scores in 
English/language arts (ELA) and 0.024 standard deviations in math. Providing equal 
access to effective teaching for FRL and non-FRL students would reduce the student 
achievement gap from 28 percentile points to 26 percentile points in ELA and from 
26 percentile points to 24 percentile points in math in a given year. In one alternative 
model specification, however, access to effective teaching for disadvantaged students 
and nondisadvantaged students was not statistically different. 1 


1 We conducted this alternative model in 9 districts for the upper elementary grades and 23 districts for the 
middle school grades where the necessary data were available. 
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• Access to effective teaching for disadvantaged students did not change over time 
in the study districts. Average differences in effective teaching between FRL and 
non-FRL students did not differ over the three study years for either ELA or math. 

• Disadvantaged students’ access to effective teaching varied across school 
districts. Access to effective teaching varied across study districts, ranging from 
districts with equal access to districts with differences in effective teaching for FRL 
and non-FRL students as large as 0.106 standard deviations of student test scores in 
ELA and 0.081 standard deviations of student test scores in math. Disadvantaged 
students did not have greater access to effective teaching in any school district in the 
sample. 

• Unequal access to effective teaching was most related to the school assignment of 
teachers and students rather than to the way that teachers were assigned to 
students within schools. The average between-school measure of access to effective 
teaching was significantly greater than the average within-school measure in both the 
upper elementary and middle school grades. Differences in effective teaching 
between schools for FRL and non-FRL students were larger than differences within 
schools by 0.020 standard deviations of student test scores in ELA and by 0.008 
standard deviations in math. In other words, unequal access to effective teaching 
depended more on FRL students attending schools with less effective teaching than 
on FRL students being assigned to classrooms (within schools) with less effective 
teaching. 

Research Questions and Study Overview 

To address the need for evidence on access to effective teaching, the U.S. Department of 
Education’s Institute of Education Sciences (IES) contracted with Mathematica Policy Research 
to study the issue in a diverse set of school districts over the five-year period from the 2008- 
2009 to the 2012-2013 school years. The study’s primary research questions are: 

1. To what extent do disadvantaged students have equal access to effective teaching 
within school districts, and how does this change over time? 

2. Is access to effective teaching related to different patterns of teacher hiring, 
retention, and mobility for high- and low-poverty schools? 

This study builds on the current evidence base in three ways. First, it documents access to 
effective teaching in districts that are diverse in terms of geography and size, with 29 districts in 
16 states and all 4 U.S. Census regions. Second, it examines whether access to effective teaching 
changes over time. In this report, we measure access to effective teaching over a three-year 
period. Ultimately, we will measure changes over a five-year period. Third, we measure the 
extent of inequities between as well as within schools, allowing us to incorporate the effects of 
both between-school sorting of students and teachers to schools and within-school matching of 
teachers to students. 

In this report, the first of three, we provide results that answer the first research question 
based on the first three years of the study (2008-2009 through 2010-2011 school years). The 
second report will address the second research question for the same school years, and the final 
report will update the results for both research questions to cover an additional two years 
(through the 2012-2013 school year). 
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Participating Districts 

To document access to effective teaching in a diverse set of districts, the recruitment and 
selection of districts focused on obtaining a geographically diverse sample that could provide the 
data needed for a value-added analysis of teachers. We sought districts with a mix of free or 
reduced-price lunch (FRL) and non-FRL students — because we measure differences in effective 
teaching between these two groups of students — and districts that implemented different types of 
policies. 


The 29 study districts are geographically diverse, with at least 4 districts from each region of 
the country. In these study districts, the percentage of students from the South and Midwest is 
similar to the national distribution, students from the North are underrepresented, and students 
from the West are overrepresented. The study districts more closely resemble the 100 largest 
districts than they resemble all districts in the United States. The study districts are large, on 
average, with a median enrollment of 60,000 students, and are located in medium-sized or large 
cities (Figure ES.l). Sixteen of the 29 study districts have more than 75 percent of students in a 
large city. The average study district has an FRL rate of 63 percent, with a range of 34 to 
78 percent. Thirty-one percent of students in study districts are Black, 40 percent are Hispanic, 
and 18 percent are English-language learners. 


Figure ES.l. Distribution of Study Districts, U.S. Districts, and Largest 100 U.S. Districts by Size 


100% 98% 



Fewer than 25,000 50,000 100,000 

25,000 to 49,999 to 99,999 or more 

students students students students 


■ All U.S. Districts 


Largest 100 
U.S. Districts 

■ Study Districts 


Source: 2008-2009 Common Core of Data. 
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Most study districts regard equitable access to effective teaching as a policy priority, but 
there is variation across districts in the types of policies they are implementing that may affect 
access to effective teaching. According to interviews with district staff that we conducted, a 
majority of our districts (17 of the 29) described equitable access to effective teaching as a policy 
priority. However, most districts (22 of 29) reported they had not used data on teacher 
effectiveness, such as value-added measures or teacher evaluation ratings, to assess access to 
effective teaching. Of the 12 policies potentially related to disadvantaged students’ access to 
effective teaching that we asked about, the most commonly reported were in the areas of school 
improvement and teacher development policies. At least half of the study districts reported using 
these policies. 

Measuring Access to Effective Teaching 

To measure whether disadvantaged students have equal access to effective teaching, we 
calculated what we refer to as the Effective Teaching Gap (ETG). The ETG is a measure that 
compares the average effectiveness of teaching experienced by nondisadvantaged students with 
the average effectiveness of teaching received by disadvantaged students. A positive ETG means 
that the typical disadvantaged student experiences or has access to less effective teaching than 
the typical nondisadvantaged student, while a negative ETG means that disadvantaged students 
experience more effective teaching. An ETG of zero indicates that disadvantaged students have 
equal access to effective teaching. 

To further understand access to effective teaching between and within schools in a district, 
we separated each district’s ETG into between-school and within-school ETGs. The district ETG 
is the sum of the between- and within-school ETGs. Access to effective teaching can differ 
between schools if disadvantaged students attend schools that have less effective teaching on 
average than those attended by nondisadvantaged students. These between-school differences are 
related to how families select schools and how teachers come to be employed — and remain 
employed — in those schools. Access to effective teaching can also differ within a given school. 
Within-school differences can occur if teacher-student assignment within schools differs 
systematically for disadvantaged versus nondisadvantaged students. 
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Study Design 

Student Sample. We examined access to effective teaching in English/language arts (ELA) and 
math among students in 29 study districts in grades 4 through 8. These are the subjects and grades for 
which test score data are available from the end of the current and prior school years. 

Data. We collected administrative data to estimate teacher-level value-added models and 
measure access to effective teaching in study districts. In particular, we collected four years of 
standardized student test scores from state assessments in grades 3 through 8, a set of student 
characteristics (FRL status, limited English proficiency, special education status, gender, race, and 
ethnicity), school enrollment data for students, and teacher-student-course links indicating the teacher 
responsible for teaching ELA and/or math to each student. We report results from the 2008-2009 
through 2010-2011 school years for 24 districts, and results from the 2007-2008 through 2009-2010 
school years for the other 5 districts where we gathered data from state databases that were lagged by 
one year. We also collected information on district policies by conducting a document search and then 
using information from the documents to inform interviews with district staff. 

Measuring Effective Teaching. To measure effective teaching, we used value-added analysis, a 
statistical approach to isolate a teacher’s contribution to student achievement. It measures the 
achievement levels of a teacher’s students after accounting for students’ prior achievement levels and 
other characteristics, such as special education or English language learner status that may be related 
to student achievement during the year. A value-added model predicts the test score each student 
would have achieved with the average teacher in a district or state, and then compares the average 
actual performance of a given teacher’s students to the average of these students' predicted scores. 
The difference between the two scores is attributed to the teacher as his or her value-added estimate. 
One critique of value added is that unmeasured differences between students could bias value-added 
estimates, but some evidence suggests that unmeasured student characteristics do not play a large 
role in determining teacher value added (Kane and Staiger 2008; Chetty et al. 2011). 

Measuring Access to Effective Teaching. We measured access to effective teaching within 
each district using the Effective Teaching Gap (ETG). We calculated the district ETG in four steps: 

Stepl: Use value-added analysis to measure the effectiveness of each teacher in the 
district. 

Step 2: Assign each student in the district the value added of his or her teacher in the 
relevant subject. This value-added estimate represents the effectiveness of teaching 
experienced by the student for a given subject. 

Step 3: Using students’ free or reduced-price lunch (FRL) status as the measure of 
disadvantage, we calculate the mean value-added estimate among all 
nondisadvantaged students in the district and conduct the same calculation among 
all disadvantaged students. 

Step 4: Calculate the district ETG by subtracting the mean value-added estimate for 
disadvantaged students from the mean value-added estimate for nondisadvantaged 
students. 

Measuring Between- and Within-School Access to Effective Teaching. We calculated the 
between-school ETG following the same steps described above for the district ETG, but we replaced 
teacher value added with the average value added of the teachers within each school, grade, subject, 
and year. For simplicity, we refer to this as the “school value-added estimate” (even though it is 
computed separately for each grade within a school). It measures the effectiveness of the average 
teacher at the school. By assigning each student (FRL or non-FRL) the school value-added estimate, 
we eliminated any differences in access to effective teaching that can arise from the assignment of 
teachers to students within a school. Thus, the between-school ETG can be calculated by taking the 
difference in average school value added between non-FRL and FRL students. Since the sum of the 
between- and within-school ETGs is the district ETG, we first calculated the between-school ETG and 
then subtracted it from the district ETG to determine the within-school ETG. 
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We focus solely on value added in this study for two reasons. First, comparisons of teacher 
effectiveness based on multiple measures are not feasible in this study because other measures of 
teacher effectiveness, such as structured observations of classroom practices, are not conducted 
in all study districts or not measured consistently across districts. This study takes advantage of 
the existing student achievement data available from districts to measure access to effective 
teaching in a consistent way across districts. Second, value added is a policy-relevant measure 
because current federal policy encourages the use of student achievement growth (of which value 
added is an example) as a significant factor in assessing teacher effectiveness. 

Access to Effective Teaching 

We examined whether disadvantaged students have equal access to effective teaching within 
districts by documenting the size of the ETG in the 29 study districts for grades 4 through 8. We 
present evidence on the ETG separately for ELA and math. 

On average in the 29 study districts and across the three school years, disadvantaged 
students did not have equal access to effective teaching. The differences in effective teaching 
for FRL and non-FRL students in a given year were equivalent to a shift of two percentile points 
in the student achievement gap. Teachers of non-FRL students had higher value added than 
teachers of FRL students on average, with statistically significant differences of 0.034 standard 
deviations of student test scores in ELA and 0.024 standard deviations in math (Figure ES.2). 
The results imply that the typical FRL student experiences less effective teaching than the typical 
non-FRL student within a district. In addition, the average ETG did not significantly differ over 
the three years of the study for either subject. 

Reducing the ETG from its current level to zero for one year — in other words, providing 
equal access to effective teaching for FRL and non-FRL students — would reduce the student 
achievement gap in the average study district from 28 to 26 percentile points in ELA and from 
26 to 24 percentile points in math. We also calculated how the average teacher of a 
nondisadvantaged student compares to the average teacher of a disadvantaged student in tenns of 
percentiles of the teacher distribution. For ELA, the average teacher of nondisadvantaged 
students is at the 56th percentile in the teacher distribution, compared to the average teacher of 
disadvantaged students at the 47th percentile. In math, it is the difference between a teacher at 
the 53rd percentile and a teacher at the 48th percentile. 

The main findings were not sensitive to calculating the ETG based on a comparison of 
students in different racial/ethnic groups rather than different FRL status. The Black/White ETG 
and Hispanic/White ETG differ from the FRL ETG by no more than 0.005 standard deviations of 
student test scores in either subject. 

We also calculated the ETG when effective teaching is based on two alternative value-added 
models, a value-added model that used an additional year of baseline test scores of students and a 
separate model that incorporated characteristics of students’ classroom peers to capture peer 
effects. In the alternative model with two years of baseline scores, disadvantaged students had 
less access to effective teaching. However, in the alternative model incorporating peer effects, 
access to effective teaching for disadvantaged and nondisadvantaged students was not 
statistically different. 
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Figure ES.2. Effective Teaching Gap in Study Districts, 29-District Average, Years 1 to 3 



(S) w 


-0.04 J 


English/Language Arts 


Source: District administrative data 


Note: Results are for 29 districts, grades 4 through 8, and years 1 to 3. District-level results are 

weighted across grades and years by the number of teachers. Overall results are weighted 
equally across districts. 

indicates statistical significance at the 0.05 level, two-tailed test. Statistical significance is based on 
variation across districts. 


Access to effective teaching for FRL and non-FRL students varied across study 
districts, with equal access to effective teaching in some districts and unequal access 
favoring non-FRL students in other districts. The ETG ranges from districts with equal access 
(ETGs not significantly different from zero) to districts with ETGs as large as 0.106 in ELA and 
0.081 in math (Figures ES.3 and ES.4). Variation in the ETGs across study districts is greater 
than would be expected to occur by chance. We found unequal access to effective teaching in 27 
of the 29 districts in ELA and in 19 of the 29 districts in math. In the remaining districts, 
disadvantaged students have equal access to effective teaching, as shown by ETGs that are not 
significantly different from zero. None of the study districts has a statistically significant ETG 
favoring FRL students. 
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Figure ES.3. Distribution of Effective Teaching Gaps in English/Language Arts, 29 Districts, Years 1 to 3 



Source: District administrative data. 

Note: Results are for 29 districts, grades 4 through 8, and years 1 to 3. ETGs are computed within each 

district-grade-year combination and averaged with equal weight across years within each district. 
Circles represent the district-level ETGs and the vertical lines show the 95 percent confidence intervals 
around each point. The cross-district average of 0.034 standard deviations is shown by the dashed 
horizontal line. Districts are ordered by the size of the ETG. District codes 1 to 29 are assigned in order 
of the size of the district ETG in English/language arts and are consistent across figures. The ETG is 
expressed in terms of standard deviations of student test scores. 


Figure ES.4. Distribution of Effective Teaching Gaps in Math, 29 Districts, Years 1 to 3 



Source: District administrative data. 

Note: Results are for 29 districts, grades 4 through 8, and years 1 to 3. ETGs are computed within each 

district-grade-year combination and averaged with equal weight across years within each district. 
Circles represent the district-level ETGs and the vertical lines show the 95 percent confidence intervals 
around each point. The cross-district average of 0.024 standard deviations is shown by the dashed 
horizontal line. Districts are ordered by the size of the ETG. District codes 1 to 29 are assigned in order 
of the size of the district ETG in English/language arts and are consistent across figures. The ETG is 
expressed in terms of standard deviations of student test scores. 
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Differences in access to effective teaching are larger between schools than within 
schools. States and districts often focus on access to effective teaching between schools but do 
not capture inequities within schools. The ETG allows us to separately measure the degree to 
which differences in effective teaching occur between schools and within schools. We find that 
most of the district ETG is accounted for by between-school differences, but some differences in 
access to effective teaching arise within schools as well. 

The between-school ETG is larger than the within-school ETG, especially in ELA at the 
elementary grades. For ELA, the between-school ETG is 0.029 standard deviations of student 
test scores larger than the within-school ETG in the upper elementary grades (grades 4 and 5) 
and 0.014 standard deviations larger in the middle school grades (Table ES.l). In addition, the 
difference in the between- and within-school ETGs is significantly larger for upper elementary 
grades than for middle school grades. District-level results for ELA are shown in Figures ES.5 
and ES.6. 

The between-school ETG for math is also significantly larger than the within-school ETG. 
However, the between-school differences in effective teaching do not explain unequal access to 
the same degree that it does in ELA. The between-school ETG is larger than the within-school 
ETG by 0.011 in the upper elementary grades and 0.005 in the middle school grades. District- 
level results for math are shown in Figures ES.7 and ES.8. 

The patterns of between- and within-school ETGs may be related to the tendency for 
elementary schools to be smaller than middle schools, resulting in a more homogenous student 
population due to less diversity in household income within smaller attendance areas. 


Table ES.l. Average Between-School and Within-School Effective Teaching Gaps, 29-District 
Average, Years 1 to 3 


Subject 

Between-School 

Within-School 

Difference 

P-Value 

English/Language Arts 

Upper Elementary 

0.035 

0.005 

0.029* 

0.00 

Middle 

0.022 

0.008 

0.014* 

0.00 

All Grades 

0.027 

0.007 

0.020* 

0.00 

Math 

Upper Elementary 

0.016 

0.005 

0.011* 

0.00 

Middle 

0.016 

0.011 

0.005* 

0.04 

All Grades 

0.016 

0.008 

0.008* 

0.00 


Source: District administrative data 

Note: Results are for 29 districts, grades 4 through 8, and years 1 to 3. Upper elementary is grades 4 and 5; 

middle school is grades 6 through 8. District-level results are weighted across grades and years by the 
number of teachers. Overall results are weighted equally across districts. The difference in the between- 
and within-school ETG is the average of the differences for individual districts. The ETG is expressed in 
terms of standard deviations of student test scores. 

‘Indicates statistical significance at the 0.05 level, two-tailed test. We test whether differences in the between- and 
within-school ETGs are statistically significant by using variation within districts. 
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Figure ES.5. Between-School and Within-School Effective Teaching Gaps in English/Language Arts by 
District for Upper Elementary Grades, 29 Districts, Years 1 to 3 
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Source: District administrative data. 

Note: Results are for 29 districts, grades 4 and 5, and years 1 to 3. The ETGs are computed within each 

district-grade-year combination and then averaged with equal weight across years within each district. 
Districts are ordered by the size of the district ETG. District codes 1 to 29 are assigned in order of the 
size of the district ETG in English/language arts and are consistent across figures. The solid bars show 
between- or within-school ETGs that are significantly different from zero at the 0.05 level; the hollow 
bars indicate between- or within-school ETGs that are not significantly different from zero. 

‘Indicates statistical significance at the 0.05 level, two-tailed test. 


Figure ES.6. Between-School and Within-School Effective Teaching Gaps in English/Language Arts by 
District for Middle School Grades, 29 Districts, Years 1 to 3 



Source: District administrative data. 

Note: Results are for 29 districts, grades 6 to 8, and years 1 to 3. The ETGs are computed within each district- 

grade-year combination and then averaged with equal weight across years within each district. Districts 
are ordered by the size of the district ETG. District codes 1 to 29 are assigned in order of the size of the 
district ETG in English/language arts and are consistent across figures. The solid bars show between- 
or within-school ETGs that are significantly different from zero at the 0.05 level; the hollow bars indicate 
between- or within-school ETGs that are not significantly different from zero. 

‘Indicates statistical significance at the 0.05 level, two-tailed test. 
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Figure ES.7. Between-School and Within-School Effective Teaching Gaps in Math by District for Upper 
Elementary Grades, 29 Districts, Years 1 to 3 



Source: District administrative data. 

Note: Results are for 29 districts, grades 4 and 5, and years 1 to 3. The ETGs are computed within each 

district-grade-year combination and then averaged with equal weight across years within each district. 
Districts are ordered by the size of the district ETG. District codes 1 to 29 are assigned in order of the 
size of the district ETG in English/language arts and are consistent across figures. The solid bars show 
between- or within-school ETGs that are significantly different from zero at the 0.05 level; the hollow 
bars indicate between- or within-school ETGs that are not significantly different from zero. 

‘Indicates statistical significance at the 0.05 level, two-tailed test. 

Figure ES.8. Between-School and Within-School Effective Teaching Gaps in Math by District for Middle 

School Grades, 29 Districts, Years 1 to 3 
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Source: District administrative data. 

Note: Results are for 29 districts, grades 6 to 8, and years 1 to 3. The ETGs are computed within each district- 

grade-year combination and then averaged with equal weight across years within each district. Districts 
are ordered by the size of the district ETG. District codes 1 to 29 are assigned in order of the size of the 
district ETG in English/language arts and are consistent across figures. The solid bars show between- 
or within-school ETGs that are significantly different from zero at the 0.05 level; the hollow bars indicate 
between- or within-school ETGs that are not significantly different from zero. 

‘Indicates statistical significance at the 0.05 level, two-tailed test. 
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